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(a) : J^naase»t& to the specifications 

Amend tim second paragraph, on page 1 to read as follows* 

Related Applications : 

This invention is related to an application entitled: 
Multiprocessor System with Dynamic Cache Coherency Regions, T3S&^ 
lQ/603,251 , filed jyn& 25, , filed concurrently herewith and 
referenced herein. 

AMend the paragraph on Page 1, containing line 19 to read as 
follows i 

The description set forth in e&eee this re lated co -pending 
application is hereby incorporated into the present application 
by this reference. 

Amend page 2, penultimate paragraph containing line 32 to r^ad as 
follows * 

Although caches have worked well for multiprocessor systems with 
a moderate number of processors, prior-art multiprocessor designs 
do not scale well when extended to large numbers of processors 
for many import ant workloads including the tra.nsact.ion and 
database workload simulated by tte a TPC-C benchmark, 

Miend page 8, after Una 19, by insertion of the following 
paragraphs ; 



-2- 



IBK-FOU920Q30D4GUS1 Exnr Jesse David. Diller, 1Q/ 6032 52 
- 3 - 



Fxgjjgg § yJOAi^Ea teg particularly how supervisor ..Software moves a 
software ..process out of one coherency region that is no longer 
going to be used by that software process and into another 
coherency region that has _been created to cover the _gafl>!g_aadres5 
space as the firs t- pu t whi ch will include a n a w_Bgt_af....pr.pceBBinp 
nodes , 

Figure ft&j^tratea particularly Inc omin g storage requests,, some 
Of which misses all of the cashes ±n an or iginator's coherency 
region and some of which h it in an. pro nator's coherency region . 

Figure 1 Q„ illustrates particularly processing nodes which are 
^M...t,0.1deDtify incoming s^r a ^x^es.U ,, which are target ii*^, 

frhafr are no longer part of ...thg,,,Mfe^ ; i Space of any software, 

prpgese tih at is currently enabled by the__supervisor software, 
aag-fliaiaaJighed on the node. 

Amend page 6 by replacing the first full, paragraph with the 
following; 

In fulfilling these determined needs , we have hardware coherency 
controls which enable a system which uses multiple cache 
coherency regions to operate without the use of cache purges 
during some operations which move software, processes between 
coherency regions . The current invention works for the case 
(Figure 8} where a software process is moved out of one coherency 
region that is no longer going to be used and into another that 
has been created, to cover the same address space as the first taut 
which will include a new set of processing nodes. The preferred 
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embodiment of our invention works to allow a supervisor program 
to move a software process from one; coherency region encompassing 
one set of processing nod.es to another coherency region 
encompassing another set of processing nodes without requiring 
cache purges of caches in any of the processing nodes, if the 
destination coherency region contains fewer hardware processing 
nodes than the original coherency region, the size of the 
coherency region has been effectively been reduced . 



Amend page 7 by replacing the first full and second paragraphs 
with the. following! 

Processing nodes which use the invention , as illustrated 
particularly with Figures ft and 10, are able to identify incoming 
storage requests which target lines that are no longer part of 
the address space of any software process that is currently 
enabled by the supervisor software to be dispatched on the node* 
In. the preferred embodiment this information allows a processing 
node to identify cache lines that are no longer actively used by 
any software processes on that node and to change the cache 
entries to invalid in response to a storage request from outside 
the coherency region . 

The advantages of the invent ion, ajt,„i,Umt r . ated part i cular 1 y 

with Figures 9 and 10, are numerous. One advantage of the 
invention is that it eliminates the need for cache control 
hardware that would otherwise he required to perform selective 
purges of caches when moving software processes between cache 
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coherency regions . A second advantage is that the invention 
allows all of the caches in a system to continue processing 
coherency transactions while the coherency boundaries for a 
software process are effectively changed. A third advantage is 
that cache lines belonging to a software process that is no 
longer actively being dispatched on a given node can be 
identified and invalidated thereby enabling their reuse. 

Anniid pag® 12 i^inairig at line 4 to read as follows: 

Alternative approaches may be found in the related patent 
application, The referenced related patent application USSN 
10/ 603, .251 uses a combination of software control programs and 
hardware mode bits to define dynamic coherency boundaries in a 
computing system which utilizes more than one processing node. 
The coherency boundaries can be adjusted to create coherency 
regions that cover any number of processing nodes,, from one node 
to all the nodes in the entire system. Th® related application 
also describes how multiple coherency regions can be defined , 
each operating on a private address space. The coherency regions 
can be expanded to include additional nodes at any time during 
system operation. The coherency regions can also be reduced in 
size by removing a processing node from that region, while 
following the required procedures. Included in those procedures 
is the need to purge some cache entries in the processing node 
which is about to be removed from the coherency region. The 
cache entries which need to be removed are only those that hold 
cached copies of main storage lines that are part of the 
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coherency region that, is being reduced! in size. Caches which are 
unable to perform a selective purge based upon the identification!. 
o£ the coherency region ttet ^owns* the cached line must be 
purged completely. The selective purges require additional cache 
controller hardware as compared to prior- art designs . The 
preferred embodiment illustrated here is applicable to a 
processing system taking advantage! of the ability to eliminate 
the need for the cache purges when moving a software process 
between two distinct sets of processing nodes, 

fbe related patent application ug,g»„, ,10/603 .,2£l describes how 
the supervisor software must change the "cache coherence mode* of 
the processor when dispatching # software process that uses a 
cache coherency 



Replace the paragraph beginning a* the last partial paragraph on 
page 13 and continuing to be the first partial paragraph on page 
14 with the following paragraph: 

Th& current invention alters the logic which, is used to decide 
which processing nodes must examine any particular storage 
request, as compared to the related patent application, in £he 
referenced related patent application the coherency region, as 
expressed by the mode bits., we*?e w&l, «sed to make the 
determination, in the current preferred enbodimeat of the 
invention ij^e^Figure $ and 10) this is changed so that any 
storage request which misses all of the caches in the 
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0r.igi5ia.t0r ' s coherency region is then sent on to all processing 
nodes in the entire system, regardless of the setting of the mode 
bits. Requests which hit in the originator's coherency region 
but which do not. have the correct cache state do not need to foe 
sent outside the coherency region. An. example of this latter 
case is a. storage request which intends to alter a line of 
storage hut which finds during the course of a storage 
transaction that a cache within its coherency region has a copy 
of the line that is marked as shared. The cache state 
transitions of the current invention are set up to ensure, that a 
cache line cannot foe marked as shared in two separate coherency 
regions. When the supervisor program is moving a coherency 
region from one set of processing nodes; to another set of 
processing nodes it is effectively leaving behind cache entries 
for the coherency region on the old nodes. The current invention 
works to ensure that these old cache entries will be seen by 
requests originating from the new processing nodes and that cache 
entries for the same main storage addresses will not be 
established in the new processing nodes until the old entries are 
inval idated. 



ftnusond the specification, by Inserting the following paragr&gihs 
after the first full paragraph of page 15 as the last paragraphs 
of the detailed descriptions 
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Mq'SrjTTn^j^^ software 
prpcegg out of one coherency ..reaion that is no longer going to be 
^ii^Jl&LJki^^ and into another coherency region 

t.bat has been created to cover the same address spa ce as the 

first but which includes a new set of ^QCBB^q^^r , h§ 

illustrated in Figure S f at step,,, 81„^ 

of process 0 on coherency ...reglgn a (^2 ^ Fi^ur,.^,,J^^ eea Ss^i^^----g-t.ep 
820 the super^ fr fo e Coherency Mode Bits for 

process Q alters the active Coherency Region Tables (ACRTs) 

JBLJtoOBflLJ^^ The Coherency Region ID 

gf .PqaBMQjyLJ^^ o f Node 2 and Node 3 

H^gim^L ljM...C5>^tmCY....EMim of process 0 is added to the 

^m^ J^M^JL lh!m...^....Shxxm^...M^ . ixi 820 allows the 

. . .. t he process 0 on a new set 
Of ,proqe£ftfrr&,^^^ .Figure 1) .. 



Mow, in re 
'tri-j iL.ii- t ' ' u-'t h i fy . 



Lasaaiaa..at^^ requests 



and 3 
„ which , target lines 




This matnod also allow^^^ 
computer system ^.mMSMKMM 
wki the. t cah^^ 

«gg*9fri.ya£^^ and invalidates 

£&£b&^AM&^^ process that is no longer 
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act ivelv being di spatcfcea w_jfcjgjjag^^ thereby 
enabling the ir rf^fg. 

Figure 3 shows ton achieved. Coherency requests are 

■f..ljg.f:.^... fe^Q-^- , 3 ! !5.afi!t only within the coherency region speeif lejg ....jgy;...fehg 
Coherency Mode Bit setting of the processo r wMcj?....9^^qiifla^^.....^. 
storage request (910) . The or i.q ina t i^q,..pgg„gjf Ci?llegfcB the 
initial responses from the other ,m;p,c es.^oxt...ia U^.. coherence 
region and determines whethgr fr he- r eques t ftff&,„missed all of the 
caches in the coherence .reqi.ot? f920} . fin v„ .storage rearuest which 
misses all of th #„J^ processor's coherency 

region is rebrpa^s.fe (jjOJ, fcja.JiAl m .PXfiffilSS5i.n<? nodes in the entire 

system, recf.ardlega,,^^^^ mode bits . 

The Coherency R^iogLm..^^ initiated the 

request is , sen t it aloj ^^^ during this second 

broadcast . if it ... if.^^^e^in^, £iX5t storage request 

hit in one of th^cach^ region the 

need to broadcast the req uest ont^i$e p£ the coherency region. 

Thus, as shown in Figure 10 when a nqde ye^eive e i an incoming 

storage request (1010) that incl&aje^^ ID, it 

tests {1020] to determine if the Coherency 

entry in the node's Active C oher eney Real on Tahl e , J£,.„ it, ....does. 

{1040} . If the Coherency Regi on id doe in 
the node^s Active Coherency Region Tablg tte^ 

determined that any cache entr y as SffSM.tf^ most 
be "left over" from a process,.. that; ,.Mi i ,teE.Mldia..,a new 

coherency region. In ll thi s^^jg [I.ffl3„a,h LllSL^ft . responds to the 

coherency re<^est....ff.^ copy and/or 

PSrfonjg^^^ The actions described in 
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IMM3...2 wjll.,.,g.&use any line that was *laft over" in the "old" 

coherency region, the coherency region of the process Jae fprj_^h£ 
process was moved, to be removed from the .^^Z^9^^.^^Y...:^.^i3^ 
af t er tha firs t , at t,eme&§&^^ 

in the new qqthjHBMK^ it has 

been moved , 

Amend tha ABSTRACT OF THE DISCLOSURE AS FOLLOWS J 

ABSTRACT OF THE DISCLOSURE : 

A multiprocessor computer system has a plura lity of processing 
nodes which use processor state information to determine which 
coherent caches AFV--the--v»y-9teFft are required to examine a coherency 
transaction produced by a single originating processor's storage 
request* A node a-gH&he-eswatHge* Ms dynamic coherency boundaries 
such that the hardware uses only a subset of the total processors 



time and can optimize cache coherency as the supervisor 
software or firmware* expands and contracts the number of 
processors which are s b e i n g used to run any single workload. 
Multiple instances of a node can be connected with a second level 
controller to create a larger multiprocessor system. The node 




determine which nodes must receive any given transaction— 
r-ee e- ived by the second level controller . Logical partitions are 
mapped to allowable physical processors. Cache coherence regions 




for a single workload at any specific point in 




■aj fty— given transaction that is received by the node 
. ■e y g- ^he occ and level controller uses- the mode bits to 
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wi « li " - e fig&ifaaa subs eta of the total number af processors and 
caches in the ay atom - are. chosen for their physical proximity, A 
distinct cache coherency region can be hyparvisor defined for 
each partition ■ 
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